xen: arm: force guest memory accesses to cacheable when MMU is disabled

xen: arm: force guest memory accesses to cacheable when MMU is disabled

On ARM guest OSes are started with MMU and Caches disables (as they are on
native) however caching is enabled in the domain running the builder and
therefore we must ensure cache consistency.

The existing solution to this problem (a0035ecc0d82 "tools: libxc: flush data
cache after loading images into guest memory") is to flush the caches after
loading the various blobs into guest RAM. However this approach has two short
comings:

 - The cache flush primitives available to userspace on arm32 are not
   sufficient for our needs.
 - There is a race between the cache flush and the unmap of the guest page
   where the processor might speculatively dirty the cache line again.

(of these the second is the more fundamental)

This patch makes use of the the hardware functionality to force all accesses
made from guest mode to be cached (the HCR.DC == default cached bit). This
means that we don't need to worry about the domain builder's writes being
cached because the guests "uncached" accesses will actually be cached.

Unfortunately the use of HCR.DC is incompatible with the guest enabling its
MMU (SCTLR.M bit). Therefore we must trap accesses to the SCTLR so that we can
detect when this happens and disable HCR.DC. This is done with the HCR.TVM
(trap virtual memory controls) bit which also causes various other registers
to be trapped, all of which can be passed straight through to the underlying
register. Once the guest has enabled its MMU we no longer need to trap so
there is no ongoing overhead. In my tests Linux makes about half a dozen
accesses to these registers before the MMU is enabled, I would expect other
OSes to behave similarly (the sequence of writes needed to setup the MMU is
pretty obvious).

Apart from this unfortunate need to trap these accesses this approach is
incompatible with guests which attempt to do DMA operations with their MMU
disabled. In practice this means guests with passthrough which we do not yet
support. Since a typical guest (including dom0) does not access devices which
require DMA until after it is fully up and running with paging enabled the
main risk is to in-guest firmware which does DMA i.e. running EFI in a guest,
with a disk passed through and booting from that disk. Since we know that dom0
is not using any such firmware and we do not support device passthrough to
guests yet we can live with this restriction. Once passthrough is implemented
this will need to be revisited.

The patch includes a couple of seemingly unrelated but necessary changes:

 - HSR_SYSREG_CRN_MASK was incorrectly defined, which happened to be benign
   with the existing set of system register we handled, but broke with the new
   ones introduced here.
 - The defines used to decode the HSR system register fields were named the
   same as the register. This breaks the accessor macros. This had gone
   unnoticed because the handling of the existing trapped registers did not
   require accessing the underlying hardware register. Rename those constants
   with an HSR_SYSREG prefix (in line with HSR_CP32/64 for 32-bit registers).

This patch has survived thousands of boot loops on a Midway system.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>